EnergyParser

This notebook shows the basic functionality of loading an IDF file and operating on it in XML.

Setup

I'm using Python 2.7.5 (default, Sep 16 2013, 23:11:01) [MSC v.1500 64 bit (AMD64)] However, I always use the "from future import division" and "from future import print_function" to make transition easy


In [1]:
print('Python ' + sys.version)


Python 2.7.5 (default, Sep 16 2013, 23:11:01) [MSC v.1500 64 bit (AMD64)]

Extensive use of logging to track what's happening each step, if this is not clear read up on logging


In [2]:
import logging
logging.basicConfig(format='%(funcName)-20s %(levelno)-3s: %(message)s', level=logging.DEBUG, datefmt='%I:%M:%S')
logging.debug('Test logging')


<module>             10 : Test logging

Here's how I'm loading the module into IPython notebook


In [3]:
sys.path.append(r'C:\EclipseWorkspace\EnergyParser')

Load the module containing the IDF class


In [4]:
import idf.idf_parser as idf
#idf.IDF?

Basic loading

Here's what's happening below:
Load an .idf file from a path on disk
the from_IDF_file is the common way to instantiate
from_IDF_file calls:

  1. load_IDF() which loads file into attribute IDF_string
  2. parse_IDF_to_XML() which parses the IDF_string into attribute XML - self.XML is a lxml tree object!
  3. A random 'ID' string is created so you can track what's happening with multiple instances of the class

In [5]:
path_test_idf = r"C:\EclipseWorkspace\EnergyParser\SampleIDFs\5ZoneElectricBaseboard.idf"
new_idf = idf.IDF.from_IDF_file(path_test_idf)


load_IDF             10 : NhXk: Loaded IDF C:\EclipseWorkspace\EnergyParser\SampleIDFs\5ZoneElectricBaseboard.idf with 3679 lines
parse_IDF_to_XML     10 : NhXk: Converted IDF to XML:<type 'lxml.etree._Element'> <Element EnergyPlus_XML at 0x3fd6c48>, 348 objects
from_IDF_file        10 : NhXk: Created an IDF object named NhXk, with 348 objects

In [6]:
print(new_idf)


IDF:NhXk, IDF Lines:3679, XML Objects:348, XML_root:<Element EnergyPlus_XML at 0x3fd6c48>

Here is some raw IDF text in the IDF_string attribute


In [7]:
for line in new_idf.IDF_string.split('\n')[987:1010]:
    print(line)


  BuildingSurface:Detailed,
    C1-1P,                   !- Name
    FLOOR,                   !- Surface Type
    CLNG-1,                  !- Construction Name
    PLENUM-1,                !- Zone Name
    Surface,                 !- Outside Boundary Condition
    C1-1,                    !- Outside Boundary Condition Object
    NoSun,                   !- Sun Exposure
    NoWind,                  !- Wind Exposure
    0.0,                     !- View Factor to Ground
    4,                       !- Number of Vertices
    26.8,3.7,2.4,  !- X,Y,Z ==> Vertex 1 {m}
    30.5,0.0,2.4,  !- X,Y,Z ==> Vertex 2 {m}
    0.0,0.0,2.4,  !- X,Y,Z ==> Vertex 3 {m}
    3.7,3.7,2.4;  !- X,Y,Z ==> Vertex 4 {m}

  BuildingSurface:Detailed,
    C2-1P,                   !- Name
    FLOOR,                   !- Surface Type
    CLNG-1,                  !- Construction Name
    PLENUM-1,                !- Zone Name
    Surface,                 !- Outside Boundary Condition
    C2-1,                    !- Outside Boundary Condition Object

And here is the raw XML tree:


In [8]:
for cnt,line in enumerate(new_idf.XML):
    print(line)
    if cnt > 8: break


<!--XML Schema for EnergyPlus version 6 'IDF' files and OpenStudio version 0.3.0 'OSM' files-->
<!--Schema created April. 2011 by Marcus Jones-->
<Element OBJECT at 0x3fb8708>
<Element OBJECT at 0x3fb8948>
<Element OBJECT at 0x3fb8908>
<Element OBJECT at 0x3fb87c8>
<Element OBJECT at 0x3fb8708>
<Element OBJECT at 0x3fb8948>
<Element OBJECT at 0x3fb8908>
<Element OBJECT at 0x3fb87c8>

Basic introspection and printing

With the core IDF class loaded, it's time to inspect and manipulate it


In [9]:
import idf.utilities_xml as util_xml

Convenience functions exist for listing objects, below zone names


In [10]:
util_xml.get_zone_name_list(new_idf)


Out[10]:
['PLENUM-1', 'SPACE1-1', 'SPACE2-1', 'SPACE3-1', 'SPACE4-1', 'SPACE5-1']

Using 'PrettyTable 0.5' module, nicely formatted summaries are possible
Below, a full listing of all classes and their names, and then a table showing the idf class and how many instances of each class exist The print_table utility function has an argument for the number of rows, remove it to show all


In [11]:
table = util_xml.get_table_all_names(new_idf)
util_xml.print_table(table,5)


+--------------------------------------------+-----------------------+
|                   Class                    |          Name         |
+--------------------------------------------+-----------------------+
|                                AirLoopHVAC | VAV Sys 1             |
|                 AirLoopHVAC:ControllerList | OA Sys 1 Controllers  |
|                 AirLoopHVAC:ControllerList | VAV Sys 1 Controllers |
|               AirLoopHVAC:OutdoorAirSystem | OA Sys 1              |
| AirLoopHVAC:OutdoorAirSystem:EquipmentList | OA Sys 1 Equipment    |
+--------------------------------------------+-----------------------+

In [12]:
table = util_xml.get_table_object_count(new_idf)
util_xml.print_table(table, 10)


+--------------------------------------------+-------+
|                   Class                    | Count |
+--------------------------------------------+-------+
|                                AirLoopHVAC | 1     |
|                 AirLoopHVAC:ControllerList | 2     |
|               AirLoopHVAC:OutdoorAirSystem | 1     |
| AirLoopHVAC:OutdoorAirSystem:EquipmentList | 1     |
|                     AirLoopHVAC:ReturnPath | 1     |
|                   AirLoopHVAC:ReturnPlenum | 1     |
|                     AirLoopHVAC:SupplyPath | 1     |
|                   AirLoopHVAC:ZoneSplitter | 1     |
|        AirTerminal:SingleDuct:VAV:NoReheat | 2     |
|          AirTerminal:SingleDuct:VAV:Reheat | 3     |
+--------------------------------------------+-------+

So we see that there are 3 'AirTerminal:SingleDuct:VAV:Reheat' objects. A selection can be made around all instances matching a class name.


In [13]:
util_xml.tree_get_class(new_idf, 'AirTerminal:SingleDuct:VAV:Reheat')


tree_get_class       10 : Search of ^AirTerminal:SingleDuct:VAV:Reheat$ 3 hits in <Element EnergyPlus_XML at 0x3fd6c48>
Out[13]:
[<Element OBJECT at 0x40f1708>,
 <Element OBJECT at 0x40f1148>,
 <Element OBJECT at 0x40f1288>]

Regular expressions are fully supported in the code (note the '^' and '$' sigils!)<br> By default, an exact '^$' match


In [14]:
selection = util_xml.tree_get_class(new_idf, 'AirTerminal', flgExact = False)
print(selection)


tree_get_class       10 : Search of AirTerminal 5 hits in <Element EnergyPlus_XML at 0x3fd6c48>
[<Element OBJECT at 0x3fe2d08>, <Element OBJECT at 0x40f1508>, <Element OBJECT at 0x40f1708>, <Element OBJECT at 0x40f1148>, <Element OBJECT at 0x40f1288>]

So we found the 2 VAV:NoReheat plus the 3 VAV:Reheat. What are they? Each selection in the list is an XML node. XML nodes can be operated on, printed, etc., according to the lxml module.


In [15]:
util_xml.printXML(selection[0])


<OBJECT>
  <CLASS>AirTerminal:SingleDuct:VAV:NoReheat</CLASS>
  <ATTR Comment="- Name">SPACE2-1 VAV System</ATTR>
  <ATTR Comment="- Availability Schedule Name">ReheatCoilAvailSched</ATTR>
  <ATTR Comment="- Air Outlet Node Name">SPACE2-1 In Node</ATTR>
  <ATTR Comment="- Air Inlet Node Name">SPACE2-1 ATU In Node</ATTR>
  <ATTR Comment="- Maximum Air Flow Rate {m3/s}">autosize</ATTR>
  <ATTR Comment="- Zone Minimum Air Flow Input Method">Constant</ATTR>
  <ATTR Comment="- Constant Minimum Air Flow Fraction">0.3</ATTR>
</OBJECT>

This is the general structure of the EnergyPlus XML schema: Each OBJECT represents an IDF object. It has a class name, and 'n' attributes. The first attribute is sometimes, but not always, the name. The parser also captures the comments in the IDF string.

Classes can be deleted. Note below that 2 objects are deleted, reducing XML Object count from 348 to 346. However, the IDF lines remain at 3679. The ASCII text representation is not reflected to XML representation until convert_XML_to_IDF is called.


In [16]:
print(new_idf)
util_xml.delete_classes(new_idf, ['AirTerminal:SingleDuct:VAV:NoReheat'])
print(new_idf)


tree_get_class       10 : Search of ^^AirTerminal:SingleDuct:VAV:NoReheat$$ 2 hits in <Element EnergyPlus_XML at 0x3fd6c48>
delete_classes       10 : NhXk: Deleted 2 ^AirTerminal:SingleDuct:VAV:NoReheat$ objects
IDF:NhXk, IDF Lines:3679, XML Objects:348, XML_root:<Element EnergyPlus_XML at 0x3fd6c48>
IDF:NhXk, IDF Lines:3679, XML Objects:346, XML_root:<Element EnergyPlus_XML at 0x3fd6c48>

convert_XML_to_IDF() uses an XLST transfrom to reproduce the IDF. Currently, comments are not written back to ASCII.


In [17]:
new_idf.convert_XML_to_IDF()
print(new_idf)


convert_XML_to_IDF   10 : NhXk: Converted XML to IDF, 346 objects
IDF:NhXk, IDF Lines:3914, XML Objects:346, XML_root:<Element EnergyPlus_XML at 0x3fd6c48>

Finally, this new object can be written back to disk. Note that write_IDF() calls convert_XML_to_IDF() first, so manual calls to convert_XML_to_IDF() are usually never necessary.


In [18]:
new_idf.write_IDF('d:\\testing EnergyParser.idf')


convert_XML_to_IDF   10 : NhXk: Converted XML to IDF, 346 objects
write_IDF            10 : NhXk: Wrote IDF d:\testing EnergyParser.idf, 346 objects

It might also be interesting to write the XML to disk directly.


In [19]:
new_idf.write_XML('d:\\testing EnergyParser.xml')


write_XML            10 : NhXk: Wrote XML d:\testing EnergyParser.xml

Advanced object manipulation

The definition of a valid IDF file is described by the Input Data Dictionary IDD file. This concept of validation is also important in XML, with the concepts of a schema (XSD) and Document Type Definition (DTD). Both describe the structure of an XML document. It could be possible to create an XSD from the IDD, and have a powerful definition tool within the XML paradigm. However this project does not support this. Instead, because the IDD has the exact same syntax as IDF, it is read directly as follows.


In [20]:
path_idd = r"D:\Apps\EnergyPlusV8-1-0\Energy+.idd"
idd_definition = idf.IDF.from_IDD_file(path_idd)


load_IDF             10 : None: Loaded IDF D:\Apps\EnergyPlusV8-1-0\Energy+.idd with 90012 lines
parse_IDF_to_XML_2   10 : None: Converted IDD to XML:<type 'lxml.etree._Element'> <Element EnergyPlus_XML at 0x40f8588>, 727 objects
from_IDD_file        10 : None: Created an IDD (DEFINITION) object named None, with 727 objects

Note that this is the from_IDD_file() method, NOT from_IDF_file()
The IDD file has a different syntax compared to IDF which describes all aspects of each object
The speed of loading can be increased by writing this back to XML and using from_XML_file

Below is an example of an IDD object converted into XML. There is signifantly more information describing each attribute of each class, all of which is captured by the parser. This is a fairly flat representation where the information is captured in XML attributes. A clearer representation would be more hierarchical, but this suffices.


In [21]:
target_class = util_xml.tree_get_class(idd_definition, 'Site:WeatherStation')[0]
util_xml.printXML(target_class)


tree_get_class       10 : Search of ^Site:WeatherStation$ 1 hits in <Element EnergyPlus_XML at 0x40f8588>
<OBJECT>
  <CLASS unique-object="" memo="This object should only be used for non-standard weather data.  Standard weather data such as TMY2, IWEC, and ASHRAE design day data are all measured at the default conditions and do not require this object.">Site:WeatherStation</CLASS>
  <ATTR field="Wind Sensor Height Above Ground" type="real" units="m" default="10.0" minimum_GT="0.0">N1</ATTR>
  <ATTR field="Wind Speed Profile Exponent" type="real" default="0.14" minimum="0.0">N2</ATTR>
  <ATTR field="Wind Speed Profile Boundary Layer Thickness" type="real" units="m" default="270.0" minimum="0.0">N3</ATTR>
  <ATTR field="Air Temperature Sensor Height Above Ground" type="real" units="m" default="1.5" minimum="0.0">N4</ATTR>
</OBJECT>

An advanced manipulation consists of defining
1) Which class
2) The specific instance name
3) The attribute to change (Retreived from IDD)
4) The new value of this attribute for all matched items
All selection criteria are full regex supported, so '.' matches to 'any' matched string
This definition is contained in a dictionary

For example, let's change the cieling height of all spaces to be 3 m. First, let's look at one of the spaces in detail;


In [22]:
selection = util_xml.tree_get_class(new_idf, 'Zone')
util_xml.printXML(selection[3])


tree_get_class       10 : Search of ^Zone$ 6 hits in <Element EnergyPlus_XML at 0x3fd6c48>
<OBJECT>
  <CLASS>Zone</CLASS>
  <ATTR Comment="- Name">SPACE3-1</ATTR>
  <ATTR Comment="- Direction of Relative North {deg}">0</ATTR>
  <ATTR Comment="- X Origin {m}">0</ATTR>
  <ATTR Comment="- Y Origin {m}">0</ATTR>
  <ATTR Comment="- Z Origin {m}">0</ATTR>
  <ATTR Comment="- Type">1</ATTR>
  <ATTR Comment="- Multiplier">1</ATTR>
  <ATTR Comment="- Ceiling Height {m}">2.438400269</ATTR>
  <ATTR Comment="- Volume {m3}">239.247360229</ATTR>
</OBJECT>

Next, get this class from the IDD (Not this IDF!)


In [23]:
target_class = util_xml.tree_get_class(idd_definition, 'Zone')[0]
print(target_class)
#util_xml.printXML(target_class)


tree_get_class       10 : Search of ^Zone$ 1 hits in <Element EnergyPlus_XML at 0x40f8588>
<Element OBJECT at 0x3fe2d08>

And get the integer position of our desired field. This is done again on the IDD object, since the IDF objects may not have this information (no comments in IDF file!).


In [24]:
util_xml.get_IDD_matched_position(target_class,'field','Ceiling Height')


get_IDD_matched_position 10 : <Element OBJECT at 0x3fe2d08> field=Ceiling Height positions [8]
Out[24]:
8

This concept can also be used to select objects with a

Now we have all information required to make a precise selection of this attribute in the IDF file. Let's select all Zone objects with the name starting with SPACE, so we don't select any PLENUM's. A utility function is provided which handles all of the above steps. It is called with a dictionary defining all aspects of the change; the class name, the object instance name (first ATTR), the attribute aka field name (From the IDD definition), and what value you want matching attributes to have.


In [40]:
this_change =  {'class'    :'^Zone$',
                'objName'  :'^SPACE',
                'attr'     :'Ceiling Height',
                'newVal'   :'3.0',
                }

In [41]:
util_xml.apply_change(new_idf, idd_definition, this_change)


apply_change         10 : NhXk: Applied change 5 times: 
{'newVal': '3.0', 'attr': 'Ceiling Height', 'class': '^Zone$', 'objName': '^SPACE'} 
Out[41]:
<idf.idf_parser.IDF at 0x3fa2240>

In [44]:
selection = util_xml.tree_get_class(new_idf, 'Zone')
for obj in selection:
    break
    util_xml.printXML(obj)


tree_get_class       10 : Search of ^Zone$ 6 hits in <Element EnergyPlus_XML at 0x3fd6c48>

This concept is flexible through Regular expressions.

Example: Exact match and update of a zone.

In [45]:
this_change =  {'class'    :'^Zone$',
                'objName'  :'^SPACE4-1$',
                'attr'     :'Ceiling Height',
                'newVal'   :'3.5',
                } 
util_xml.apply_change(new_idf, idd_definition, this_change)


apply_change         10 : NhXk: Applied change 1 times: 
{'newVal': '3.5', 'attr': 'Ceiling Height', 'class': '^Zone$', 'objName': '^SPACE4-1$'} 
Out[45]:
<idf.idf_parser.IDF at 0x3fa2240>
Example: Match all zones with any name using regex wildcard.

In [48]:
this_change =  {'class'    :'^Zone$',
                'objName'  :'.',
                'attr'     :'Ceiling Height',
                'newVal'   :'2.8',
                } 
util_xml.apply_change(new_idf, idd_definition, this_change)


apply_change         10 : NhXk: Applied change 6 times: 
{'newVal': '2.8', 'attr': 'Ceiling Height', 'class': '^Zone$', 'objName': '.'} 
Out[48]:
<idf.idf_parser.IDF at 0x3fa2240>

In general my use case is as follows;

  1. Define n variants with m 'change definition dictionaries'
  2. Load IDD
  3. Loop over n:
    1. Load IDF
    2. Make deletions of unwanted classes
    3. Merge in other useful bits of IDF code i.e. ASHRAE rotations, output variables, etc.
    4. Loop over m change definition dictionaries
    5. Write the nth IDF back to disk

Using Excel or a text file etc., the above dictionary structures can be listed in tables to define a workflow.


In [ ]: